Accurate face-modeling has extensive applications in areas such as human computer interaction, multimedia, and face recognition. In recent years, a number of approaches have been proposed for 3D face-modeling from images. For example, large angle multiple views have been conventionally used for accurately recovering shape information. But such systems are manually intensive and far from flexible, since the user has to manually specify point matching and feature correspondences across multiple images.
One approach is based on a morphable 3D face model. The approach obtains face model reconstruction from a single image, which demonstrates the advantage of using models of the linear class. Because of the sensitivity of the texture descriptor to illumination change, however, the quality of shape reconstruction degrades with uncontrolled illumination. The texture descriptor can be replaced by pair-wise point matching to somewhat increase robustness to illumination change.
Model-based bundle adjustment techniques enhance results. In model-based bundle adjustment, prior model knowledge is included into the traditional bundle adjustment. When applied to face-modeling, the 3D shape can be reliably recovered, but relies on a sparse face mesh structure, which is not a sufficient representation of real face geometry. A similar bundle adjustment procedure is based on the sophisticated PCA model learned from real 3D face data. Given pair-wise feature correspondences as input, the approach is robust to uncontrolled lighting conditions. The precision of the reconstructed face model can be predicted as a function of the number and quality of the correspondences.
However, the computational complexity of the above technique increases cubically with the number of frames being processed, which makes it infeasible for processing a relatively long video sequence. In addition, the approach depends on the quality of point matching between adjacent frames, which is unreliable in low quality video. What is needed is an efficient technique for automatic recovery of accurate 3D face models from videos captured by a low cost camera.